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ABSTRACT 

The  phonological  structure  of  human  languages  is  intricate,  yet  highly 
constrained.  Through  a  combination  of  connectionist  modeling  and 
linguistic  analysis,  we  are  attempting  to  develop  a  computational  basis 
fcv  the  nature  of  phonology.  We  present  a  connectionist  architecture 
that  performs  multiple  simultaneous  insertion,  deletion,  and  mutation 
operations  on  sequences  of  phonemes,  and  introduce  a  novel  additional 
primitive,  clustering.  Clustering  provides  an  interesting  alternative  to 
both  iterative  and  relaxation  accounts  of  assimilation  processes  such  as 
vowel  harmony.  Our  resulting  model  is  efficient  because  it  processes 
utterances  entirely  in  parallel  using  only  feed-forward  circuitry. 

1  INTRODUCTION 

Phonological  phenomena  can  be  quite  complex,  but  human  phonological  behavior  is 
also  highly  constrained.  Many  operations  that  are  easily  learned  by  a  perception-like 
sequence  mapping  network  are  excluded  from  real  languages.  For  example,  as  Pinker 
and  Prince  (1988)  point  out  in  their  critique  of  the  Rumelhart  and  McClelland  (1986) 
verb  learning  model,  human  languages  never  reverse  the  sequence  of  segments  in  a 
word,  but  this  is  an  easy  mapping  for  a  network  to  learn.  On  the  other  hand,  we  note  that 
some  phonological  processes  that  are  relatively  common  in  human  languages,  such  as 
vowel  harmony,  appear  difficult  for  a  sequence-mapping  architecture  to  learn.  Why  are 
only  certain  types  of  sequence  operations  found  in  human  languages,  and  not  others?  We 
suggest  that  this  is  a  reflection  of  the  limitations  of  an  underlying,  genetically-determined, 
specialized  computing  architecture.  We  are  searching  for  this  architecture. 
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Our  woric  was  initially  inspired  by  Oec^e  Lakey’s  theory  of  cognitive  phonology 
(Lakoff,  1988,  1989).  which  is  in  turn  a  development  of  the  ideas  (tf  John  Goldsmith 
(to  appear).  Lakoff  proposes  a  three-level  representation  scheme.  The  M  (moipho- 
phonemic)  level  rqjresents  the  underlying  form  of  an  utterance,  the  P  (phonemic)  level 
is  an  intennediate  fmm,  and  the  F  (phortedc)  level  is  the  derived  surface  form. 

Lakoff  uses  a  combination  of  inter-level  mapping  rules  and  intra-level  well-formedness 
conditions  to  specify  the  relationships  between  P-  and  F-level  representations  and  the 
M-level  input  In  a  comectionist  implementation,  the  computations  performed  by  the 
mapping  rules  are  straightforward,  but  we  find  the  well-formedness  conditions  troubling. 
Goldsmitl/s  proposal  was  that  phonology  is  a  goal-directed  constraint  satisfaction  sys¬ 
tem  that  operates  via  parallel  relaxation.  He  cites  Smolensky’s  harmony  theory'  LakoH 
has  adopt^  this  appeal  to  harmony  theory  in  his  description  of  how  well-formedness 
conditions  could  work. 

In  our  model,  we  further  develop  the  Goldmsitis  and  Lakoff  mapping  scheme,  but  we  rejea 
harmony-based  well-formedness  conditions  for  several  reasons.  First,  harmony  theory 
involves  simulated  annealing  search.  The  timing  constraints  of  real  nervous  systems  rule 
out  simulated  annealing.  Second,  it  is  not  clear  how  to  construct  an  energy  function  for 
a  connectionist  network  that  performs  complex  discrete  phonological  operations.  FinaUy 
there  is  our  desire  to  explain  why  certain  types  of  processes  occur  in  human  languages 
and  others  do  not  Harmony  theory  alone  is  too  unconstrained  for  this  purpose. 

We  have  implemented  a  model  called  M^P  (for  “Many  Maps”  Model  of  Phonology)  that 
allows  us  to  account  for  virtually  all  of  the  phenomena  in  (Lakoff,  1989)  using  a  tightly- 
constrained,  purely-feedforward  computing  scheme.  In  the  next  section  we  describe  the 
mailing  matrix  architecture  that  is  the  heart  of  M^P.  Next  we  give  an  example  of  an 
iterative  process,  Yawelmani  vowel  harmony,^,  which  Lakoff  models  with  a  P-level  well- 
formedness  condition.  Such  a  condition  would  have  to  be  implemented  by  relaxation 
*  search  for  a  “minimum  energy  state”  in  the  P-level  rqnesentation,  which  we  wish  to 
avoid.  Finally  we  unseat  our  alternative  approach  to  vowel  harmony,  using  a  novel 
clustering  mechanism  that  eliminates  the  need  for  relaxation. 

2  THE  MAPPING  MATRIX  ARCHITECTURE 

Figure  1  is  an  overview  of  our  “many  maps”  model.  M-P  constructions  compute  how 
to  go  from  the  M-level  representation  of  an  utterance  to  the  P-level  representation.  The 
derivation  is  described  as  a  set  of  explicit  changes  to  the  M-level  string.  M-P  construc¬ 
tions  read  the  segments  in  the  M-level  buffer  and  write  the  changes,  phrased  as  mutation, 
deletion,  and  insertion  requests,  into  slots  of  a  buffer  called  P-deriv.  The  M-level  and 
P-deriv  buffers  are  then  read  by  the  M-P  mapping  matrix,  which  produces  the  P-level 
represeniiation  as  its  output  The  process  is  repeated  at  the  next  level,  with  P-F  con¬ 
structions  writing  changes  into  an  F-deriv  buffer,  and  a  P-F  map  deriving  an  F-level 

‘  Srooleadcy'i  “hannony  theory”  thould  not  be  confuted  with  ihelinguinic  phenomenon  of  “vowel  haimony.” 

^Yiwelnuni  it  ■  dialect  of  Yokuu,  an  American  Indian  language  from  California.  Our  Yawelmani  data  it 
drawn  from  Kenttowicz  and  Kitieberth  (1979),  at  it  Lakoff’t. 
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Figure  1:  Overview  of  the  “many  maps”  model. 


representation.  A  final  stq)  called  “canonicalization”  cleans  up  the  rqnesentations  of  the 
individual  segments. 

Figure  2  shows  the  effect  of  an  M-P  construction  that  breaks  iq)  CCC  consonant  clusters 
by  inserting  a  vowel  after  the  first  consonant,  producing  CiCC.  The  input  in  this  case 
is  the  Yawelmani  word  /?ugnhin/  “drinks**,  and  the  desired  insertion  is  indicated  in  P- 
deriv.  The  mtqiping  matrix  derives  the  P-level  reimsentation  right>justified  in  the  bufler, 
with  no  segment  gaps  or  collisions.  It  can  do  this  even  when  muddle  simultaneous 
insertions  and  deletions  are  being  performed.  But  it  cannot  perform  arbitrary  sequence 
manipulations,  such  as  reversing  all  the  segments  of  an  utterance.  Further  details  of  the 
matrix  architecture  are  given  in  (Touretzky,  1989)  and  (Wheeler  and  Touretzky,  1989). 

3  ITERATIVE  PHENOMENA 


Several  types  of  phonological  processes  operate  on  groups  of  adjacent  segments,  often  by 
making  them  more  similar  to  an  immediately  preceding  (or  following)  trigger  segment. 
Vowel  harmony  and  voicing  assimilation  are  two  examples.  In  Yawelmani,  vowel  har¬ 
mony  takes  the  following  form:  an  [ahigh]  vowel  that  is  preceded  by  an  [^high]  round 
vowel  becomes  round  and  back.  In  the  form  /dois+al/  “might  report",  the  non-round, 
back  vowel  /a/  is  [-high],  as  is  the  preceding  round  vowel  /o/.  Therefore  the  /a/  becomes 
round,  yielding  the  surface  form  [do:sol].  Similarly,  in  /dul>t-hin/  “leads  by  the  hand”,  the 
[■t-high]  vowel  /i/  is  preceded  by  the  [-t-high]  round  vowel  Ai/,  so  the  A/  becomes  round  and 
back,  giving  [dubhun].  In  /bok*-f-hin/  “finds**,  the  /i/  does  not  undergo  harmony  because 

it  differs  in  height  from  the  preceding  vowel.  _ 
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Harmony  is  described  as  an  iterative  process  because  it  can  tqjply  to  entire  sequences  of 
vowels,  as  in  the  following  derivation: 

A’ul+sit+hin/  “bums  for" 

/t’ul+sut-f-hin/  harmony  on  second  vowel 
A’ul+sut+hun/  harmony  on  third  vowel 

In  Yawelmani  we  saw  an  epenthesis  process  that  inserts  a  high  vowel  A/  to  break  up 
lengthy  consonant  clusten.  Epenthetic  voweb  may  either  undergo  or  block  harmony. 
With  the  word  /logw+xsj  “let’s  pulverize",  epenthesis  inserts  an  /i/  to  break  up  the  /gwV 
cluster,  producing  /logiw+xa/.  Now  the  /a/  is  preceded  by  a  [-t-high,  -round]  vowel,  so 
harmony  does  not  apply,  whereas  in  /do:s-fal/.  which  has  the  same  sequeiKe  of  underlying 
vowels,  it  did.  This  is  an  instance  of  epenthesis  blocking  harmony.  In  other  environments 
the  epenthetic  vowel  may  itself  undergo  harmony.  For  example: 

/?ugn+hin/  “drinks" 

/?uginhin/  epenthesis 

/?ugunhin/  harmony  on  epenthetic  vowel 

/?ugunhut)/  harmony  on  third  vowel 

The  standard  generative  phonology  analysis  of  harmony  utilizes  the  following  rule,  ap¬ 
plying  after  epenthesis,  that  is  supposed  to  iterate  through  the  utterance  from  left  to  right, 
changing  one  vowel  at  a  time; 
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Lakoff  oflen  an  alternative  account  of  qjentbesis  and  harmony  that  eliminates  iteration. 
He  states  q)enthesis  as  an  M-P  construction: 

M:  C  C  {C.#} 

I  I  I 

P:  t]  i  [] 

The  harmony  rule  is  stated  as  a  P-level  well-formedness  condition  that  applies  simulta¬ 
neously  throughout  the  buffer 

P:  If  [+syll,  around,  ahigh]  Co  X, 

then  if  X  s  [-t-syll.  ahi^],  then  X  «  [-fround,  -t-back]. 

Starting  with  /?ugn-fhin/  at  M-Ievel,  Lakoff’s  model  would  settle  into  a  representation 
of  /?ugimhun/  at  P-level.  We  rq>eat  again  the  crticial  point  that  this  representation  is 
not  derived  by  sequential  applic^n  of  rules;  it  is  merely  licensed  by  one  plication 
of  qienthesis  and  two  (tf  harmony.  The  actual  computation  of  the  P-level  representation 
would  be  performed  by  a  parallel  relaxatitm  process,  perhaps  using  simulated  annealing, 
that  somehow  determines  the  sequence  that  best  satisfies  all  applicable  constraints  at 
P-levcl. 

4  THE  CLUSTERING  MECHANISM 

Our  account  of  vowel  harmony  must  differ  from  Lakoffs  because  we  do  not  wish  to 
rely  on  relajtation  in  our  model  Instead,  we  introduce  special  clustering  circuitry  to 
recognize  sequences  of  segments  that  share  certain  properties.  The  clustering  idea  is 
meant  to  be  analogous  to  perceptual  grouping  to  vision.  Sequences  of  adjacent  visually- 
similar  objects  ate  naturally  perceived  as  a  whole.  A  similar  mechanism  operating  on 
phonological  sequences,  although  unprecedented  in  linguistic  theory,  does  not  ^tpear 
implausible.  Crucial  to  our  model  is  the  principle  that  perceived  sequences  may  be 
operated  on  as  a  unit.  This  allows  us  to  avoid  iteration  and  give  a  fuUy-patallel  account 
of  vowel  harmony. 

The  clustering  mechanism  is  controlled  by  a  smaU  number  of  language-specific  param- 
eten.  The  rule  shown  below  is  the  P-F  clustering  rule  for  Yawelmani  Cluster  type 
[■(■syllabic]  indicates  that  the  rule  looks  only  u  vowels.  (This  is  implemented  by  an 
additional  mapping  matrix  that  extracts  the  vowel  projection  of  the  P-level  buffer.  The 
clustering  mechanism  actually  looks  at  the  output  of  this  matrix  rather  than  at  the  P-level 
buffer  directly.)  The  trigger  of  a  cluster  is  a  round  vowel  of  a  given  height,  and  the 
elements  are  the  subsequent  adjacent  vowels  of  matching  height  Application  of  the  rule 
causes  elements  (but  not  triggers)  to  undergo  a  change;  in  this  case,  they  become  round 
and  back. 
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Yawelmani  vowel  harmony  —  P-F  mapping: 

Cluster  type:  [4-syllabic] 

Trigger  [around,  ahigh] 

Element  [ahigh] 

Change:  [-t-tound,  -t-back] 

The  following  hypothetical  vowel  sequence  illustrates  the  application  of  this  clustering 
rule.  Consonants  are  omitted  for  clarity: 

123456789 
i  u  i  i  e  0  o  a  i 

trigger  +  + 

element  +  +  +  + 

The  second  vowel  is  round,  so  it’s  a  trigger.  Since  the  third  and  fourth  vowels  match  it 
in  height,  they  become  elements.  The  fifth  vowel  is  [-high],  so  it  is  not  included  in  the 
cluster.  The  sixth  vowel  triggen  a  new  cluster  because  it’s  round;  it  is  also  [-high].  The 
seventh  and  ei^th  vowels  are  also  [-high],  so  they  can  be  elements,  but  the  nindi  vowel 
is  excluded  firom  the  cluster  because  is  [4-high].  Note  that  vowel  7  is  an  element,  but 
it  also  meets  the  specification  fa  a  trigger.  Given  a  choice,  our  model  prefen  to  mark 
segments  as  elements  rather  than  triggen  because  only  elements  undergo  the  specified 
change.  The  distinction  is  moot  in  Yawlemani,  where  triggen  are  already  round  and 
back,  but  it  matten  in  other  languages:  see  (Wheeler  and  Touretzky,  1989)  for  details. 

Figures  2  and  3  together  show  the  derivation  (tf  the  Yawelmani  w(»d  [?ugunhun]  from 
the  underlying  form  /?ugn-»-hin/.  In  figure  2  an  M-P  construction  inserted  a  high  vowel 
In  figure  3  the  P-F  clustering  circuitry  has  examined  the  P-level  buffer  and  marked  the 
triggen  and  elements.  Segments  that  were  marked  as  elements  then  have  the  change 
[•fround,  -fback]  written  into  their  corresponding  mutation  slots  in  F-deiiv.  Finally,  the 
P-F  mapping  matrix  produces  the  sequence  /7ugunhun/  as  the  F-level  rqnesentation  of 
the  utterance. 

5  DISCUSSION 

We  could  not  justify  the  extra  circuitry  required  for  clustering  if  it  were  suitable  only 
for  Yawelmani  vowel  harmony.  The  same  mechanism  handles  a  variety  of  other  iterative 
phenomena,  including  Slovak  and  Gidabal  vowel  shortening,  Icelandic  umlaut,  and  Rus¬ 
sian  voicing  assimilation.  The  full  mechanism  has  some  additional  parameten  beyond 
those  covered  in  the  discussion  of  Yawelmani.  For  example,  clustering  may  proceed  firom 
right-to-left  (as  is  the  case  in  Russian)  instead  of  fiom  left-to-righl  Also,  clusters  may 
be  of  either  boutKled  or  unbounded  length.  Bouttded  clusten  ate  required  for  alternation 
processes,  such  as  Gidabal  shortening.  They  cover  exactly  two  segments:  a  trigger  and 
one  element  We  are  making  a  deliberate  analogy  here  with  metrical  phonology  (stress 
systems),  where  unbounded  feet  may  be  of  arbitrary  length,  but  bounded  feet  always 
contain  exactly  two  syllables.  No  language  has  strictly  trisyllabic  feel  We  predict  a  sim¬ 
ilar  constraint  will  hold  for  iterative  phenomena  when  they  are  reformulated  in  parallel 
clustering  terms,  i.e.,  no  language  requires  bounded-length  clusters  with  more  than  one 
element 
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Figure  3:  Clustering  applied  to  Yawelmani  vowel  hannony. 


Our  model  makes  many  other  predictions  of  constninu  on  human  phonology,  based 
on  limitations  of  the  hi^ly-structured  "many  maps”  architecture.  We  are  attempting  to 
verify  these  predictions,  and  also  to  extend  the  model  to  additional  aspects  of  phonological 
behavior,  su^  as  syllabification  and  stress. 
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